fix(scan): preserve snapshot id for empty data evolution index plans - #258
fix(scan): preserve snapshot id for empty data evolution index plans#258wangyong9999 wants to merge 7 commits into
Conversation
aeeb00a to
cd817b9
Compare
…view-fixes-20260901 # Conflicts: # src/paimon/core/table/source/data_evolution_batch_scan.cpp
| snapshot_reader_->GetSnapshotManager(); | ||
| PAIMON_ASSIGN_OR_RAISE(std::optional<Snapshot> snapshot, | ||
| ResolveGlobalIndexScanSnapshot(core_options_, snapshot_manager)); | ||
| if (!snapshot) { |
There was a problem hiding this comment.
If global_index_result_ is set externally, how do we infer its snapshot ID? In that case, using ResolveGlobalIndexScanSnapshot does not seem very appropriate.
I’d suggest being stricter here: if predicates are present with global index, we should either reject time travel options such as timestamp/tag, or properly implement the corresponding support like java, rather than adding defensive checks in multiple places.
There was a problem hiding this comment.
Fixed in 7400330. Supplied empty results now take the selected snapshot from normal batch planning; we no longer infer it through a separate resolver. Tag/timestamp are rejected before evaluating a Global Index candidate, and the shared scanner change is removed.
2 participants
Purpose
Linked issue: none.
A global-index miss returned an empty plan without the snapshot ID used by the scan.
Internal index results: keep the snapshot ID used to evaluate the Global Index and attach it to
an empty plan. If latest advances before data planning, fail instead of combining index ranges and
data splits from different snapshots.
Externally supplied empty results: do not infer snapshot provenance from the result. Run normal
batch planning to resolve and validate the selected data snapshot, then retain that snapshot ID and
drop the splits.
Unsupported time travel: when a supplied Global Index result is present, or a non-partition
predicate may use the Global Index, reject effective tag- and timestamp-based startup modes before
index evaluation. Explicit snapshot IDs and latest modes remain supported; missing selectors keep
the normal
Invaliderrors.This adds no shared state, public API, ABI, storage-format, or protocol changes.
Tests
paimon-global-index-test: 138/138 passed.paimon-core-test: 1874/1874 passed.selector precedence, missing or invalid snapshot selection, strict tag/timestamp rejection, and
normal time travel when no Global Index path is requested.
git diff --checkpassed.API and Format
None.
Documentation
None.
Generative AI tooling
Generated-by: OpenAI Codex (GPT-5)